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Abstract: We discuss some aspects of the extension to continuous 
systems of a statistical measure of complexity introduced by Lopez-Ruiz, 
Mancini and Calbet (LMC) [Phys. Lett. A 209 (1995) 321]. In general, 
the extension of a magnitude from the discrete to the continuous case 
is not a trivial process and requires some choice. In the present study, 
several possibilities appear available. One of them is examined in detail. 
Some interesting properties desirable for any magnitude of complexity 
are discovered on this particular extension. 
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1 Introduction 



In the last years many complexity measures have been proposed as in- 
dicators of the complex behaviour found in different systems scattered 
in a broad spectrum of fields. Some of them come from physics such as 
the effective measure of complexity Jl], the thermodynamical depth 
and the simple measure of complexity ||. Other attempts arise from 
the field of computational sciences such as algorithmic complexity ||, |5|] , 
Lempel-Ziv complexity || and e-machine complexity 0. Other works 
try to enlighten this question in many other contexts: ecology, genet- 
ics, economy, etc., for instance, the complexity of a system based on its 
diversity || and the physical complexity of genomes || . 

Most of these proposals coincide in using concepts such as entropy 
(in physics) or information (in computational sciences) as a basic ingre- 
dient for quantifying the complexity of a phenomenon. Also there is 
a general belief that the notion of complexity in physics must start by 
considering the perfect crystal and the isolated ideal gas as examples 
of simple models with zero complexity. Both systems are extrema in 
an entropy/information scale and therefore some fundamental ingredient 
would be missing if one insisted on describing complexity only with the 
ordinary information or entropy. 

It seems reasonable to adopt some kind of distance to the equiparti- 
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tion, the disequilibrium of the system, as a new ingredient for defining 
an indicator of complexity. Going back to the two former examples, it is 
readily seen that they are extremes in a disequilibrium scale and therefore 
disequilibrium cannot be directly associated with complexity. 



The recently introduced Lopez-Ruiz-Mancini-Calbet (LMC) statisti- 
cal measure of complexity identifies the entropy or information stored 
in a system and its distance to the equilibrium probability distribution 
(the disequilibrium) as the two ingredients giving the correct asymptotic 
properties of a well-behaved measure of complexity. In fact, it vanishes 
both for completely ordered and for completely random systems. Besides 
giving the main features of an intuitive notion of complexity, it has been 
shown that LMC complexity successfully enables us to discern situations 
regarded as complex in discrete systems out of equilibrium: one instance 
of a local transition to chaos via intermittency in the logistic map | 10| , 



the dynamical behaviour of this quantity in a simplified isolated gas [11] 
and another example of classical statistical mechanics pT 



A possible formula of LMC complexity for continuous systems was 
suggested in Lopez- Ruiz et al. fllPf . Anteneodo and Plastino (JTSj pointed 
out some peculiarities concerning such an extension for continuous prob- 
ability distributions. It is the aim of this work to offer a discussion of the 
extension of LMC complexity for continuous systems. A slightly modi- 
fied extension brings interesting and very striking properties, and some 
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of the Anteneodo and Plastino remarks are solved with the new proposed 
definition. 

In Section 2 the extension of information and disequilibrium concepts 
for the continuous case are discussed. In Section 3 the LMC measure of 
complexity is rewieved and possible extensions for continuous systems 
are suggested. We proceed to present some properties of one of these 
extensions in Section 4. Finally, we establish the Conclusions. 



2 Entropy /Information and Disequilibrium 

Depending on the necessary conditions to fulfill, the extension of an es- 
tablished formula from the discrete to the continuous case always requires 
a careful study and in many situations some kind of choice between sev- 
eral possibilities. Next we carry out this process for the entropy and 
disequilibrium formulas. 



2.1 Entropy or Information 



Given a discrete probability distribution {p^} 



t=l,2,...JV 



satisfying > 



and YliLiPi — 1; the Boltzmann-Gibss-Shannon formula [14] that ac- 



counts for the entropy or information, H, stored in a system is defined 
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by 

N 

H ({Pi}) = -k^PilogPi , (1) 

i=i 

where k is a positive constant. Some properties of this quantity are: (i) 
positivity: H > for any arbitrary set {pi}, (ii) concavity: H is concave 
for arbitrary {p^} and reaches the extremal value for equiprobability (pi = 
1/N Vi), (iii) additivity: H(AU B) = H{A) + #(5) where A and 5 are 
two independent systems, and (iv) continuity: H is continuous for each of 
its arguments. And vice versa, it has been shown that the only function 
of {pi} verifying the latter properties is given by Eq. ([]]) flTJ, [15| . For an 
isolated system, the irreversibility property is also verified, that is, the 
time derivative of H is positive, dH/dt > 0, reaching the equality only 
for equilibrium. 

Calculation of H for a continuous probability distribution p(x), with 
support on [—L,L] and J^ L p(x)dx = 1, can be performed by dividing 
the interval [— L, L] in small equal- length pieces Ax = x% — % = 
1, • • • , n, with xq = —L and x n = L, and by considering the approximated 
discrete distribution {p^ = {p(xi)Ax}, i = 1, • • • ,n, with Xi a point in 
the segment It gives us 

H* = H({p t }) = (2) 

n n 

= — k ^2p(xi) logp(xj) Ax — ky^^p(xj) log(Ao;) Ax. 
i=i i=i 

The second adding term of H* in the expression (fj) grows as log n when 
n goes to infinity. Therefore it seems reasonable to take just the first and 
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finite adding term of H* as the extension of H to the continuous case: 
H(p(x)). It characterizes with a finite number the information contained 
in a continuous distribution p(x). In the limit n — > oo, we obtain 



H(p(x)) = lim 



n—»oo 



-/c^p(xi)logp(xi) A 



»=i 



= —k J p[x) \ogp(x) dx. (3) 

If p(x) > 1 in some region, the entropy defined by Eq. (^j can be- 
come negative. Although this situation is mathematically possible and 
coherent, it is unfounded from a physical point of view. See |16| for a 
discussion on this point. Let f(p, q) be a probability distribution in phase 
space with coordinates (p,q), f > and dpdq having the dimension of 
an action. In this case the volume element is dpdq/h with h the Planck 
constant. Suppose that H(f) < 0. Because of f (dpdq/h) f = 1, the ex- 
tent of the region where / > 1 must be smaller than h. Hence a negative 
classical entropy arises if one tries to localize a particle in phase space in 
a region smaller than h, that is, if the uncertainty relation is violated. In 
consequence, not every classical probability distribution can be observed 
in nature. The condition H(f) = could give us the mininal width that 
is physically allowed for the distribution and so the maximal localization 
of the system under study. This cutting property has been used in the 
calculations performed in Ref. [|P^| . 
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2.2 Disequilibrium 



Given a discrete probability distribution {pi\i=\^,...,N satisfying Pi > 
and Yh=iPi — 1, its Disequilibrium, D, is the quadratic distance of the 
actual probability distribution {p^} to equiprobability: 

D is maximal for fully regular systems and vanishes for completely ran- 
dom ones. 

In the continuous case with support on the interval [— L, L], the rect- 
angular function p(x) = 1/(2L), with — L < x < L, is the natural 
extension of the equiprobability distribution of the discrete case. The 
disequilibrium could be defined as 




If we redefine D omitting the constant adding term in D*, the disequi- 
librium reads now: 

D(p(x)) = J p 2 (x)dx. (5) 

D > for every distribution and it is minimal for the rectangular function 
which represents the equipartition. D does also tend to infinity when the 
width of p(x) narrows strongly and becomes extremely peaked. 



7 



3 Statistical Measure of Complexity 



LMC complexity, C, has been denned [JTOj as the interplay between the 
information, H, stored in a system and its disequilibrium, D. Calculation 
of C for a discrete distribution {pi}, with Pi > and i = 1, • • • , N, is 
given by the formula 



(6) 



C({ Pl }) = H({p t }) ■ D({p t }) = 

= ~ k ■ (E (Pi ~ Jj) J ■ 

This definition fits the intuitive arguments and verifies the required asymp 
totic properties: it vanishes for completely ordered systems and for fully 
random systems. C has been successfully calculated in different systems 
out of equilibrium: one instance of a local transition to chaos in an unidi- 
mensional mapping jUj, the time evolution of C for a simplified model of 



an isolated gas, the 'tetrahedral' gas [11], some statistical features of the 



behaviour of LMC complexity for DNA sequences and a modifica- 
tion of C as an effective method to identify the complexity in hydrological 
systems [fL8| . 



Feldman and Cruchtfield |19[ presented as a main drawback that C 
vanishes and it is not an extensive variable for finite-memory regular 
Markov chains when the system size increases. This is not the general 
behaviour of C in the thermodynamic limit as it has been suggested 
by Calbet and Lopez- Ruiz 0]. On the one hand, when iV — > oo and 
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k = 1/log N, LMC complexity is not a trivial function of the entropy, in 
the sense that for a given H there exists a range of complexities between 
and C max (H), where C max is given by 

[C max {H)} N ^ = H.{l-Hf . (7) 

Observe that in this case H is normalized, < H < 1, because k = 
1/log AT. On the other hand, non-extensitivity cannot be considered as 
an obstacle since it is nowadays well known that there exists a variety of 
physical systems for which the classical statistical mechanics seems to be 
inadequate and for which an alternative non-extensive thermodynamics 
is being hailed as a possible basis of a theoretical framework appropriate 
to deal with them [p0| . 

According to the discussion in Section [|, the expression of C for the 
case of a continuum number of states, x, with support on the interval 
[— L, L] and J^ L p(x) dx — 1, is defined by 

C(p(x)) = H(p(x)) ■ D(p(x)) = 

= (— k J p{x) \ogp{x) dx j • [J p 2 {x)dx\ . (8) 



Anteneodo and Plastino |13| pointed out that C can become negative. 
Obviously, C < implies H < 0. Although this situation is coherent 
from a mathematical point of view, it is not physically possible. Hence 
a negative entropy means to localize a system in phase space into a re- 
gion smaller than h (Planck constant) and this would imply to violate 
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the uncertainty principle (see discussion of Section |2.1| ). Then a distri- 
bution can broaden without any limit but it cannot become extremely 
peaked. The condition H = could indicate the minimal width that 
p(x) is allowed to have. Similarly to the discrete case, C is positive for 
any situation and vanishes both for an extreme localization and for the 
most widely derealization embodied by the equiprobability distribution. 
Thus, LMC complexity can be straightforwardly calculated for any con- 
tinuous distribution by Eq. ®. It has been applied, for instance, for 



quantifying C in a simplified two- level laser model in Ref. [12 



Anyway, the positivity of C for every distribution in the continuous 
case can be recovered by taking the exponential of H. If we define H = 
e H , we obtain a new expression C of the statistical measure of complexity 
given by 

C{p{x)) = H{p{x)) ■ D{p{x)) = e H{p{x)) ■ D{p{x)) . (9) 

In addition to the positivity, C encloses other interesting properties that 
we describe in the next section. 



4 Properties of C 



The quantity C given by Eq. @ has been presented as one of the possible 
extensions of the LMC complexity for continuous systems. We proceed 
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now to present some of the properties that characterize such a complexity 
indicator. 



4.1 Invariance under translations and rescaling trans- 
formations 

If p(x) is a density function defined on the real axis R, f R p(x) dx = 1, 
and a > and (3 are two real numbers, we denote by p a ^(x) the new 
probability distribution obtained by the action of a /3-translation and an 
a-rescaling transformation on p(x), 



When a < 1, p a ,p{. x ) broadens whereas if a > 1 it becomes more peaked. 
Observe that p a A x ) is also a density function. After making the change 
of variable y = a(x — (3) we obtain 



(10) 




The behaviour of H under the transformation given by Eq. (jT^) is the 
following: 




H{p) — log a . 



Then, 



H{p a ,p) = e 



Hip) 



a 
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It is straightforward to see that D{p a ^) = aD(p), and to conclude that 
C(p a , p ) = H( Pa ,fs) ■ D(p a>f} ) = otD{p) = C{p) . (11) 

Observe that translations and rescaling transformations keep also the 
shape of the distributions. Then it could be reasonable to denominate 
the invariant quantity C as the shape complexity of the family formed by 
a distribution p(x) and its transformed p a ^{x). Hence, for instance, the 
rectangular Il(x), the isosceles-triangle shaped A(x), the gaussian 
or the exponential S(x) distributions continue to belong to the same II, 
A, T or H family, respectively, after applying the transformations defined 
by Eq. (fT0|). Calculation of C on these distribution families gives us 

C(U) = 1 

(7(A) = -Vew 1.0991 
3 

C(T) = y| « 1.1658 
C(E) = e - w 1.3591 . 

Remark that the family of rectangular distributions has a smaller C than 
the rest of distributions. This fact is true for every distribution and it 
will be proved in Section [O. 



4.2 Invariance under replication 



Lloyd and Pagels recommend that a complexity measure should re- 
main essentially unchanged under replication. We show now that C is 
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replicant invariant, that is, the shape complexity of m replicas of a given 
distribution is equal to the shape complexity of the original one. 



Suppose p(x) a compactly supported density function, J^pix) dx = 
1. Take n copies p m (x), m — 1, • • • , n, of p(x), 

p m (x) = -^= p(Vn(x - A m )) , l<m<n, 



where the supports of all the p m (x), centered at \' m s points, m — 1, • • • , n, 
are all disjoint. Observe that /^Pmfx) dx = ^, what make the union 

n 

i=i 

to be also a normalized probability distribution, J^° OQ q(x)dx = 1. For 
every p m (x), a straightforward calculation shows that 

H(p m ) = - H{p) + - log 
ra n 

£>(Pm) = -A=^(P)- 

Taking into account that the m replicas are supported on disjoint inter- 
vals on R, we obtain 

H(q) = H(p) + log y/E, 
D {q) = 1^). 

Then, 

(7(g) = (7(p) , (12) 
what completes the proof of the replicant invariance of C. 
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4.3 Near- Continuity 



Continuity is a desirable property of an indicator of complexity. For a 
given scale of observation, similar systems should have a similar complex- 
ity. In the continuous case, similarity between density functions defined 
on a common support suggests that they take close values almost every- 
where. More strictly speaking, let 5 be a positive real number. It will 
be said that two density functions f(x) and g(x) defined on the interval 
I G R are 5 -neighboring functions on / if the Lebesgue measure of the 
points x & I verifying | f(x) —g(x) |> 5 is zero. A real map T defined on 
density functions on / will be called near- continuous if for any e > there 
exists 5(e) > such that if f(x) and g(x) are (^-neighboring functions on 
I then | T(f) - T(g) \< e. 

It can be shown that the information H, the disequilibrium D and 
the shape complexity C are near-continuous maps on the space of density 
functions defined on a compact support. We must stress at this point 
the importance of the compactness condition of the support in order to 
have near-continuity. Take, for instance, the density function defined on 
the interval [— 1,L], 

(1-5 if -l<x<0 
g 5 , L (x) = 1 | if < x < L (13) 
[ otherwise , 

with < 5 < 1 and L > 1. If we calculate H and D for this distribution 
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we obtain 



H( 9 s,l) = -(l-5)\og(l-5)-5\og[j- 



D{9s,l) 



Consider also the rectangular density function 



X[-i,o]( x ) 



1 if - 1 < x < 
otherwise . 



(14) 



If < 5 < 5 < 1, gs,L(x) and X[-i,o]( x ) are (^-neighboring functions. When 
5 — > 0, we have that lim^o 9s,l(x) = X[-i,o](x)- In this limit process the 
support is maintained and near-continuity manifests itself as following, 



\im C(g S;L ) 

o— >U 



C(X[-i,o]) = 1 ■ 



(15) 



But if we allow the support L to become infinitely large, the compactness 
condition is not verified and, although lim^oo gs^ix) and X[-i,o]( x ) are 
(^-neighboring distributions, we have that 



lim C(g s , L ) 



OO 



^ C( X[ -i, 0] ) = 1 ■ 



(16) 



Then near-continuity in the map C is lost due to the non-compactness 
of the support when L — > oo. This example suggests that the shape 
complexity C is near-continuous on compact supports and this property 
will be rigorously proved elsewhere. 
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4.4 The minimal shape complexity 

If we calculate C on the example given by Eq. ([13|) , we can verify that the 
shape complexity can be as large as wanted. Take, for instance, 8 = ~. 
The measure C reads now 

C(gs=i, L ) = \Vl (l + -jr) . (17) 

Thus C becomes infinitely large after taking the limits L — > or L — > oo. 
Remark that even in the case gs^i has a finite support, C is not upper 

bounded. The density functions, <7(a=i) ) (£_ + o) anc ^ 5 , (5=i),(L-*oo)> °f 
finitely increasing complexity have two zones with different probabilities. 
In the case L —>■ there is a narrow zone where probability rises to infin- 
ity and in the case L —>■ oo there exists an increasingly large zone where 
probability tends to zero. Both kind of density functions show a similar 
pattern to distributions of maximal LMC complexity in the discrete case, 
where there is an state of dominating probability and the rest of states 
have the same probability. 

The minimal C given by Eq. (|17D is found when L — 1, that is, when 
gs,L becomes the rectangular density function X[-i,i]- m ^ ac ^' ^ ne vame 
C = 1 is the minimum of possible shape complexities and it is reached 
only on the rectangular distributions. We sketch now some steps that 
prove this result. 
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Suppose 

f=ib^xE k (is) 

k=l 

to be a density function consisting of several rectangular pieces E k , 
k = l,---,n, on disjoint intervals. If is the Lebesgue measure of 
E k , calculation of C gives 



71 1 11 * 



k=l \k=l 

Lagrange multipliers method is used to find the real vector (/ii, /z 2 , • • • , 
Ai, A2, • • • , A n ) that makes extremal the quantity C(f) under the con- 
dition Sfc =1 Afc/ifc = 1. This is equivalent to studying the extrema of 
logC(f). We define the function z(\ k , fx k ) = logC(f)+a (Efc=i A*./i& - 1), 
then 

n / n \ / n \ 

«(Afc, yU/c) = - E ^* log A fc + log lj2 fa** + a E - 1 ■ 
fc=l \fc=l / \fc=l / 

Differentiating this expression and making the result equal to zero we 

obtain 

— = log A fc - //fc + =; n T2+a/i fc = (19) 

= -X k log\ k + — — + a\ k = (20) 

Dividing Eq. fll9"|) by fi k and Eq. (|20"D by A& we get 

2Afc 



v „ +a-l=logA fe 
— + a = log X k . 



A A: 
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Solving these two equations for every A& we have 

n 
3=1 

Therefore / is a rectangular function taking the same value A for every 
interval Ek, that is, / is the rectangular density function 

1 1 



/ = A • xl with A 



where L is the Lebesgue measure of the support. 

Then C(f) = 1 is the minimal value for a density function composed 
of several rectangular pieces because, as we know for the example given 
by Eq. (|T7|), C(f) is not upper bounded for this kind of distributions. 



Furthermore, for every compactly supported density function g and 
for every e > 0, it can be shown that near-continuity of C allows to find 



a ^-neighboring density function / of the type given by expression (18 
verifying | C(f) —C(g) |< e. The arbitrariness of the election of e brings 
us to conclude that C(g) > 1 for every probability distribution g. Thus, 
we can conclude that the minimal value of C is 1 and it is reached only 
by the rectangular density functions. 
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5 Conclusions 



Complexity theory of discrete systems has been equipped with a new 
function that not only vanishes for perfectly ordered and disordered sys- 
tems but also has resulted helpful in detecting complexity in patterns 
produced by a process. Thus LMC complexity has been shown very use- 
ful to quantify complex behaviour in local transitions to chaos in discrete 
mappings flQfl , it has permitted us to advance the concept of maximum 



complexity path in the field of systems far from equilibrium |TTJ, and, fur- 
thermore, a first attempt to quantify complexity in a model of a two-level 
laser system was performed in Ref. \T3( . 



Another remarkable feature of LMC complexity is the extension C 
to the continuous case. Results found in the discrete and the continuous 
case are consistent: extreme values of C are observed for distributions 
characterized by a peak superimposed to a uniform sea. Other merits of 
this extension have been studied and explained in the present work. 



First, we find that this quantity is invariant under translations and 
rescaling transformations. C does not change if the scale of the system is 
modified but its shape is maintained. It has been calculated on different 
families of distributions invariant under those transformations. The re- 
sult allows us to consider C as a new parameter that characterizes every 
family of distributions. 
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Second, it seems reasonable and intuitive that the complexity of m 
replicas of a given system should be the same to the original one. We 
show that C embodies this property and it is invariant under replication. 

Third, continuity is not an evident property for such a map C. Thus 
the compactness of the support of the distributions is an important re- 
quirement in order to have similar complexity for neighboring distribu- 
tions. This condition has been strictly established and stood out with an 
example. 

Finally, complexity should be minimal when the system has reached 
equipartition. We demonstrate that the minimum of C is found on the 
rectangular density functions. Its value is C — 1. Moreover, C is not an 
upper bounded function and it can become infinitely large. 

We believe and we hope that the present discussion on the extension 
of LMC complexity to the continuous case may trigger some practical 
future considerations in the area of complex systems theory. 
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